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Abstract. Consider a complete communication network of n nodes, where the nodes receive a 
common clock pulse. We study the synchronous c-counting problem: given any starting state and 
up to / faulty nodes with arbitrary behaviour, the task is to eventually have all correct nodes 
counting modulo c in agreement. Thus, we are considering algorithms that are self-stabilizing 
despite Byzantine failures. In this work, we give new algorithms for the synchronous counting 
problem that (1) are deterministic, (2) have linear stabilisation time in /, (3) use a small 
number of states, and (4) achieve almost-optimal resilience. Prior algorithms either resort to 
randomisation, use a large number of states, or have poor resilience. In particular, we achieve 
an exponential improvement in the space complexity of deterministic algorithms, while still 
achieving linear stabilisation time and almost-linear resilience. 



1 Introduction 


In this work, we design space-efficient, self-stabilising, Byzantine fault-tolerant algorithms for 
the synchronous counting problem. We are given a complete communication network on n nodes, 
with arbitrary initial states. There are up to / faulty nodes. The task is to synchronise the 
nodes so that all non-faulty nodes will count rounds modulo c in agreement. For example, here 
is a possible execution for n = 4 nodes, / = 1 faulty node, and counting modulo c = 3; the 
execution stabilises after t = 5 rounds: 

Stabilisation Counting 

Node 1: 22020012012... 

Node 2: 02010012012... 

Node 3: faulty node, arbitrary behaviour 

Node 4: 00202012012... 

Synchronous counting is a coordination primitive that can be used e.g. in large integrated 
circuits to synchronise subsystems so that we can easily implement mutual exelusion and time 
division multiple access in a fault-tolerant manner. Note that in this context it is natural to 
assume that a synchronous clock signal is available, but the clocking system usually will not 
provide explicit round numbers. Solving synchronous counting thus enables us to construct 
highly dependable round numbers for subcircuits. 

Counting modulo c = 2 is closely related to binary eonsensus: given a synchronous counting 
algorithm one can design a binary consensus algorithm and vice versa [2, 4, 5] —in particular, 
many lower bounds on binary consensus apply here as well [3, 8, 9]. However, the existing 
implementations of counting from consensus incur a factor-/ overhead in space and message 
size, rendering them very costly even in small systems. 

Prior Work. It is fairly easy to design space-efficient randomised algorithms for synchronous 
counting [5-7]: as a simple example, the nodes can just pick random states until a clear majority 
of them has the same state, after which they start to follow the majority. However, it is much 
more challenging to come up with space-efficient deterministic algorithms for synchronous 
counting [2, 4, 5], and it remains open to what extent randomisation helps in designing space- 
efficient algorithms that stabilise quickly. Fast-stabilising algorithms build on a connection 
between Byzantine consensus and synchronous counting, but require a large number of states 
per node [2, 5]. 

For small values of the parameters (e.g., n = 4 and / = 1) the synchronous counting problem 
is amenable to algorithm synthesis: it is possible to use computers to automatically design both 
space-efficient and time-efficient deterministic algorithms for synchronous counting. For example, 
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Table 1: Summary of synchronous 2-counting algorithms. For randomised algorithms, we list 
the expected stabilisation time. In [5], further trade-offs between n, the stabilization time, and 
the number of state bits are given (for / = 1). 
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there is a computer-designed algorithm that solves the case of n > 4 and / = 1 with only 3 
states per node, and another algorithm that solves the case of n > 6 and / = 1 with only 2 
states per node; both of these are optimal [4, 5]. Unfortunately, this approach does not scale, as 
the space of possible algorithms for given parameters n and / grows rapidly with re. 

In summary, currently no algorithms for synchronous counting are known that simultaneously 
scale well in terms of resilience /, stabilisation time t, and memory requirements. Here, it is 
worth noting that existing solutions with large memory requirements in essence run up to H(/) 
concurrent instances of consensus, which implies that the respective overhead extends to message 
size and the amount of local computations as well. 

Contributions. Our main contribution is a recursive construction that shows how to “amplify” 
the resilience of a synchronous counting algorithm. Given a synchronous counter for some values 
of re and /, we will show how to design synchronous counters for larger values of re and /, with 
a very small increase in time and space complexity. This has two direct applications: 

1. From a practical perspective, we can use existing computer-designed algorithms (e.g. re = 4 
and / = 1) as a building block in order to design efficient deterministic algorithms for a 
moderate number of nodes (e.g., re = 36 and / = 7). 

2. From a theoretical perspective, we can now design deterministic algorithms for synchronous 

counting for any re and for / < faulty nodes, with a stabilisation time of 0{f), and 

with only 0(log^ //loglog/) bits of state per node. 

The space complexity is an exponential improvement over prior work, and the stabilization time 
is asymptotically optimal for deterministic algorithms [8] . A summary of the related work and 
our contributions is given in Table 1. 

In our deterministic algorithms, each node broadcasts its state to all other nodes in each 
round. However, the small number of state bits bears the promise that the communication 
load can be reduced further. To substantiate the conjecture that Ending algorithms with small 
state complexity may lead to highly communication-efficient solutions, we proceed to consider a 
slightly stronger synchronous pulling model. In this model, a node may send a request to another 
node and receive a response in a single round, based on the state of the responding node at 
the beginning of the round. The cost for the exchange is then attributed to the pulling node; 
in a circuit, this translates to each node being assigned an energy budget that it uses to “pay” 
for the communication it triggers. In this model, it is straightforward to combine our recursive 
construction with random sampling to obtain the following results; 

1. We can achieve the properties of the deterministic algorithm with each node pulling 

polylog re messages in each round. The price is that the resulting algorithm retains a 
probability of to fail in each round even after stabilisation. 

2. If the failing nodes are chosen independently of the algorithm, we can fix the random 

choices. This results in a pseudo-random algorithm which stabilises with a probability of 
1 — re“ P°hiog n pggg jjggpg counting correctly. 

Structure. This paper is organised as follows. In Section 2 we formally define the model 
of computing and the synchronous counting problem. Section 3 gives the main technical 
result—a construction for creating a synchronous counting algorithm of larger resilience from 
several copies of an algorithm with smaller resilience. Section 4 uses this construction to derive 
deterministic synchronous counting algorithms with linear stabilisation time and polylogarithmic 
space complexity. Finally, in Section 5, we discuss the pulling model and how randomised 
sampling can be used to reduce the total number of communicated bits. 
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2 Preliminaries 


Model of Computation. We consider a fully-connected distributed message-passing sys¬ 
tem consisting of n processors, also called nodes, with unique identifiers from the set [n] = 
{0,1,..., n — 1}. The computation proceeds in synchronous communication rounds, where in 
each round each processor: 

1 . broadcasts its local state to all processors, 

2 . receives a vector of messages (that is, states) from all other processors, and 

3. updates its local state according to the received messages. 

However, the initial state of every node is arbitrary. Moreover, up to / nodes may be Byzantine, 
i.e., exhibit arbitrary behaviour, including to send different messages to every node. Thus, 
different nodes may receive different vectors depending on what the Byzantine nodes do. 

Algorithms. A deterministic algorithm in this model is a tuple A = {X,g, h), where X is 
the set of all possible states for a node, g: [n] x X'^ —)• A is the state transition function, and 
h: [n] X A —)• [c] maps the internal state of a node to an output value. That is, when node 
i G [n] receives a vector x G A” of messages, it will update its internal state to g{i, x) = s and 
output h{i,s) G [c]. 

The space complexity S{A) of an algorithm A is the total number of bits required to store 
the state of a node. That is, ^(A) = [log | A|]. 

Executions. For any given set T C [n] of faulty nodes, we define a projection ttj- as follows: 
for any received message vector x G X'^, let t^tIx) be a vector e such that e* = * if i G A and 
Ci = Xi otherwise. We call 7 rjr(A"') = { 7 rjr(x) : x G X^} the set of configurations. That is, a 
configuration consists only of the state of all non-faulty nodes. 

We say that a configuration d is reachable from configuration e if for every non-faulty node 
i ^ J- there exists some x G X'^ satisfying ' 7 rjr(x) = d and g{i,x) = di. Intuitively, this means 
that the Byzantine nodes can send node i such messages that i chooses to switch to state di 
when the system is in configuration e. 

An execution of a given algorithm A for a given set of faulty nodes, is an infinite sequence of 
configurations ^ = (cq, ei, 62 ,...) such that e^+i is reachable from e^. 

Synchronous Counters. We say that an execution ^ of algorithm A = {X,g, h) stabilises in 
time t if there is some ro > 0 so that for every non-faulty node i G [n] the output satisfies 

h{i, xt+r,i) = r — ro mod c for all r > 0 . 

That is, within t steps all non-faulty nodes agree on a common output and start incrementing 
their counters modulo c each round. Moreover, we say that A is a synchronous c-counter with 
resilience f if there exists a t such that for every A C [n], |A| < / all executions of A stabilise 
in t rounds. We say that the stabilisation time of algorithm A is A(A) < t. In the following, we 
use A{n, /, c) to denote the family of synchronous c-counters that run on n nodes. 

3 Boosting Resilience 

In this section, we show that given a family of synchronous c-counters for a small number of 
nodes n and resilience /, we can construct a new family of C-counters of a larger resilience 
without increasing the number of nodes in the network or the stabilisation time by too much. 
More precisely, we prove the following main result. 
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Theorem 1. Given n, / G N, pick new parameters N,F,C G N, where 

• the number of nodes is N = kn for some number of blocks 3 < A: G N, 

• the resilience is F < (/ + l)m, where we abbreviate m = \k/2'], and 

• the counter size is C > 1. 

Choose any c G N that is a multiple of 3{F + 2)(2m)^. Then for any A G A{n, f, c) there exists 
a B G A{N^ F, C) with 

r(B) < T(A) + 3{F + 2)(2m)^, 

5(B) = 5(A)+ riog(C +1)1+1. 

3.1 High-level Idea of the Construction 

Given a suitable counter A G A(n,/, c), we construct a larger network with N = kn nodes 
and divide the nodes in k blocks of n nodes. Each block runs a copy of A that is used to 
(1) determine a “leader block” and (2) output a consistent round counter within a block. 

Due to the bound on F, only a minority of the blocks (fewer than m) contains more than / 
faulty nodes. Thus, the counters of a majority of the blocks stabilise within T{A) rounds. By 
dividing the counters within each block, we let them “point” to one of m possible leader blocks 
for a fairly large number of consecutive rounds. Block i switches through leaders by a factor of 
2m faster than block i + 1. This ensures that, eventually, all stabilised counters point to the 
same leading block for 3{F + 2) rounds. 

Using a majority vote on the pointers to leader blocks, we can make sure that all correct 
nodes—also those in blocks with more than / faulty nodes—will recognise the same block as the 
leader for sufficiently long. In fact, eventually this will happen for each of the m blocks that may 
become leaders, one of which must have fewer than / faulty nodes. In particular, its counter 
will be stabilised and can be “read” by all other nodes based on taking the majority value 
within that block. Hence, we can guarantee that at some point all nodes agree on a common 
counter value for at least 3{F + 2) rounds. This value is used to control an execution of the 
well-known phase king protocol [1], which solves consensus in 0(F) rounds in systems of A > 3F 
nodes. As the constraint F < {f l)m also ensures that F < A/3, we can use the phase king 
protocol to let all nodes agree on the current value of the C-counter that is to be computed. It 
is straightforward to guarantee that, once this agreement is achieved, it will not be lost again 
and the counter is incremented by one modulo C in each round. 

3.2 Setup 

Our goal is to construct a synchronous C-counter that runs in a network oi N = kn nodes and 
tolerates F < (/ + l)m faults. We divide the nodes into k blocks, each of size n. We identify 
each node v G [kn] with a tuple (i, j) G [/c] x [n]. Thus, node v = (i, j) is the jth. node of 
block i. Blocks that contain more than / faulty nodes are said to be faulty. Otherwise, a block 
is non-faulty. 

We will have each block i run a synchronous counter A* constructed as follows. Dehne 
r = 3(F + 2) and Cj = T(2m)*+^ for i G [fe]. Let A = {X,g,h) G A{n,f,c) be the given 
synchronous c-counter, where c = ar(2m)^ for some integer a. Now for any i G [A:], we obtain a 
synchronous Cj-counter Aj = {X,g, hi) by defining the output function as hi{x) = h{x) mod Ci. 
That is, Aj outputs the output of A modulo Cj. It follows that T{Ai) = T{A) and 5(Aj) = 5(A). 
Thus, if block i is non-faulty, then Aj will stabilise in T(A) rounds. 

We interpret the value of the counter of (non-faulty, stabilised) block i as a tuple (r, y) G 
[r] X [(2m)*+^], where r is incremented by one modulo r each round, and y is incremented by 
one whenever r “overflows” to 0. We refer to the counter value that node {i,j) currently has 
according to Aj as (r[f, j],j]). Note that this value can be directly inferred from its state. 
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so by broadcasting its state {i,j) implicitly announces its current counter value to all other 
nodes. We stress that there is no guarantee whatsoever on how these variables behave in faulty 
blocks, even for non-faulty nodes in faulty blocks—the only guarantee is that non-faulty nodes 
in non-faulty blocks will count correctly after round T{A). 

We define the short-hand 


b[i,j] 


y[i,j] 

(2m)* 


mod m. 


The value b[i,j] indicates the block that the nodes in block i currently consider to be the “leader 
block” by interpreting the counter given by Aj appropriately. 

Let bq[i,j] and rq[i,j] denote the values of b[i,j] and r[i,j] in round q. We will now show 
that after stabilisation, a non-faulty block i will within c* rounds point to every block /3 G [m] 
for at least r consecutive rounds. For notational convenience, we let c_i = r. 


Lemma 1. Let i G [k] be a non-faulty block and t > T{A). For any /? G [m], there exists some 
t<w<t-\-Ci — Ci-i such that if {i,j) is non-faulty, then bq[i,j] = (3 for w < q < w -\- Ci-i. 

Proof. By round t, the counter Aj has stabilised and all non-faulty nodes (i,j) agree. First 
observe that since Aj is a Cj-counter, b[i,j] cycles through the set [m] twice in Cj rounds. Moreover, 
once b[i,j] changes its value, it will keep the new value for Cj-i = T(2m)* consecutive rounds. 

More formally, let {i,j) be a non-faulty node and u > t he the minimal u such that 
= 0 and yu[i,j] G {0,Ci/(2r)}. In particular, at round u we have bu[i,j] = 0. Moreover, 
u < t + Cil2. Since b[i,j] retains the same value for Cj_i consecutive rounds, we get that at 
round w = u + {/3 — l)ci_i the value b[i,j] changes to /3. Now we have that = /? for 

w < w' < w + Cj-i. A simple check confirms that w -|- Cj_i <t + Ci. □ 


Using this lemma, we can show that after stabilisation, all non-faulty nodes in non-faulty 
blocks will within Ck rounds point to each (3 G [m] simultaneously for at least r rounds. 

Lemma 2. Let t > T{A) and (3 G [m]. There exists some t<u<t + Ck — t such that if {i,j) 
is a non-faulty node in a non-faulty block, then bq[i,j] = (3 for u < q < u -\- t. 

Proof. We prove the statement for the special case that all blocks are non-faulty. As there is no 
interaction between the algorithm Aj of different blocks, the the lemma then follows by simply 
excluding all nodes in faulty blocks. We prove the following claim using induction; see Figure 1 
for illustration. 

Claim: For any round t > T{A), block j3 G [m] and h G [k], there exists a round t < 
u{h) < t 3- Cfi — T such that for all non-faulty nodes {i,j) where i < h we have bq[i,j] = ft for 
u{h) < q < u{h) -|- r. 

In particular, the lemma follows from the case h = k oi the above claim. For the base case 
of the induction, observe that case h = 0 follows from Lemma 1 as c_i = r. For the inductive 
step, suppose the claim holds for some /i G [A: — 1] and consider the non-faulty block h + 1. By 
Lemma 1 there exists t < w < t 3- Ch+i — Ch such that bq[h -|- 1, j] = ft for dl\ w < q < w -\- Ch 
where {h -\- l,j) is a non-faulty node. Applying the induction hypothesis to w we get u(h). 
Setting uifh -f 1) = u{h) yields that uifh -\-3)<w-\-Ch — T<t-\- Ch+i — t. This proves the claim 
and the lemma follows. □ 


3.3 Voting Blocks 

Next, we define the voting scheme for all the blocks. Define the majority operation for every 
message vector x G as follows: 

{ a if a is contained in x more than kn/2 times, 

* otherwise. 
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Block h + 2 
Block h + 1 
Block h 



t 


U U + Ch-l t + Ch 


time 


Figure 1: Counters in non-faulty blocks will eventually coincide. The picture illustrates the 
output of b[-] for three blocks i G {h, h+1, h + 2} running r(2m)*'*'^-counters with base (2m) = 6. 
For every (3 G [m], we can find a interval (blue segments) where all non-faulty blocks point to 
the same value (3 for sufficiently long, even though the clocks may cycle at different time points. 

where the symbol * indicates that the function may evaluate to an arbitrary value, including 
different values at different non-faulty nodes. We use the following short-hands as local variables: 

6* = majority{6[i, j] | j G [n]}, 

B = majority{6* | 

R = majority{r[B, j] | j G [n]}. 

Note that these functions can be locally computed from the received state vectors by checking 
for a majority and defaulting to, e.g., 0, when no such majority is found: by definition, the 
majority function may return an arbitrary value if kn/2 or fewer correct nodes “vote” for the 
same value; as non-faulty nodes broadcast the same state to all nodes, there can only be one 
such majority value. 

In words, 5® denotes the block which the nodes in block i support as a leader; different correct 
nodes may “observe” different values of 6® only if i is a faulty block or Aj has not yet stabilised. 
As F < (f + l)m = (/ + 1) [A:/2], a majority of the blocks is non-faulty. Hence, if all non-faulty 
blocks support the same leader block j3 G [m], then B evaluates to (3 at all correct nodes. By 
Lemma 2, this is bound to happen eventually. Finally, R denotes the round counter of block B, 
which is “read” correctly by all non-faulty nodes if B is non-faulty. 

Analogously to before, let i' G [k] and denote by Bq[i,j] and Rq[i,j], respectively, the values 
to which the above functions evaluate in round q at node (f,j). Then we can conclude from 
Lemma 2 that eventually all non-faulty nodes agree on R for r rounds. 

Lemma 3. There is a round t < T(A) + — t such that: 

(a) Rq[i,j] = Rq[i',j'] for any t < q < t + t and non-faulty nodes {i,j) and (^^ j’O- 

(b) Rq^i[i,j] = Rq[i,j] -|- 1 mod r for any t<q<t-\-T — 1 and non-faulty node 

Proof. As F < (f -\- l)m, there is a non-faulty block (3 G [m]. By applying Lemma 2 to round 
T(A), there is a round t < T(A) -\- Ck — t such that bq[i,j] = f3 for each t < q < t -\- t and 
non-faulty node (i, j) in a non-faulty block i. Therefore, b'q = 13 for all non-faulty blocks i. As 
F < {f + l)m, the number of faulty blocks is at most m — 1 = \k/2'] — 1 < k/2, and thus, the 
majority vote yields B = (3. 

Since T(A) < t < q < t -\- t, we have that block j3 has stabilised by round t and therefore, 
for all non-faulty nodes {(3,j), {13, j') in the non-faulty block (3. Moreover, 
as block /3 is non-faulty, it contains at most / faulty nodes. In particular, it must be that 
/ < n/3 < n/2 as otherwise counting cannot be solved and A G A{n, /, c). Hence, a majority 
vote yields R = rq[l3,j], where (/3, j) is any non-faulty node in block /3 proving claim (a). 


6 




























Set 

Instructions 

he- 

1. If fewer than N — F nodes sent a[u], set a[u] •(— 00. 

2. increment a [u]. 

he-i-T- 

1. Let Zj = {u £ [N] : a[u] = j} be the number of j values received. 

2. If Za[v] > N — F, set d[u] ■£- 1. Otherwise, set d[v] ■£- 0. 

3. Set a[u] £- min{j : Zj > F}. 

4. increment a [u]. 

he-e2- 

1. If a[u] = 00 or d[v] = 0, then set a[v] £- minjC, a[I']}. 

2. Set d[v] £- 1 and increment a[u]. 


Table 2: The instruction sets for node v £ [N] in the phase king. 


To show (b), observe that since block (3 is non-faulty and has stabilised by round t we have 
that Rq[i,j] = rq[/3,j']. Moreover, non-faulty nodes in block j3 increment r\j3,j\ by one modulo 
T in the considered interval. □ 

3.4 Executing the Phase King 

We have now built a voting scheme that allows the nodes to eventually agree on common counter 
for T rounds. Roughly speaking, what remains is to use this common counter to control a 
non-self-stabilizing T-resilient C-counting algorithm. 

We require that this algorithm guarantees two properties. First, all non-faulty nodes reach 
agreement and start counting correctly within r rounds provided that the underlying round 
counter is consistent. Second, if all non-faulty nodes agree on the output, then the agreement 
persists regardless of the round counter’s value. It turns out that a straightforward adaptation 
of the classic phase king protocol [ 1 ] does the job. 

From now on, we refer to nodes by their indices v £ [N]. The phase king protocol (like 
any consensus protocol) requires that F < N/3. It is easy to verify that this follows from the 
preconditions of Theorem 1. 

Denote by a[v] £ [C] U {oo} the output register of the algorithm, where oo is used as a “reset 
state”. There is also an auxiliary register d[u] £ {0,1}. Define the following short-hand for the 
increment operation modulo (7: 

I a[u] ■(— a[u] -|- 1 mod C if a[v\ / oo, 
increment a[v\ = < 

I no action if a[u] = oo. 

For i € [T -|- 2], we define the instruction sets listed in Table 2. First, we show that if 
these instructions are executed in the right order by all non-faulty nodes for a non-faulty leader 
i £ [F + 2], then agreement on a counter value is established. 

Lemma 4. Suppose that for some non-faulty node i £ [F -|- 2] and a round q, all non-faulty 
nodes execute instruction sets I^i, I 31 + 1 , and l 3£+2 in rounds q — 2 , q — 1, and q, respectively. 
Then aq+i[v] = aq+i\u] 7^ 00 for any two non-faulty nodes u,v £ [A^]. Moreover, dg+i[u] = 1 at 
each non-faulty node. 

Proof. This is essentially the correctness proof for the phase king algorithm. Without loss of 
generality, we can assume that the number of faulty nodes is exactly F. By assumption, we have 
F < N/3 and hence 2{N — 2F) > N — F. It follows that it is not possible that two non-faulty 
nodes v,u£ [7/] satisfy both aq-i[v], aq-i[u] £ [C] and aq-i[v] 7 ^ aq-i[u]: this would imply that 
there are at least N — 2F non-faulty nodes w that had aq- 2 [w] -f 1 mod C = aq-i[v] and the 
same number of non-faulty nodes w' with aq- 2 [w'\ -f 1 mod C = aq-i[u\, however, there are only 
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N — F < 2{N — 2F) non-faulty nodes. Therefore, there is some x G [C\ so that aq_i[?;] G {x, 00 } 
for all non-faulty nodes v. Checking / 3£+2 and exploiting that 2(N — F) > N — F once more, 
we see that this implies that also aq[v] G {x -|- 1 mod C, 00 } for some any non-faulty node v. 

We need to consider two cases. In the first case, all non-faulty nodes execute the first 
instruction of 13^+2 in round q. Then aq+i[u] = minjC, aq[£]} -|- 1 mod C for any non-faulty 
node V. In the second case, there is some node v not executing the first instruction of 13^+2 • 
Hence, dg[v] = 1, implying that v computed Za^_j^[v] > N — F in round q — I- Consequently, at 
least N — 2F > F non-faulty nodes u satisfy aq-i[u] = ag_i[u]. We infer that ag[u] = x' for all 
non-faulty nodes v: the third instruction of 13^+1 must evaluate to x' G [C] at all non-faulty 
nodes. Clearly, this implies that aq+i[u] = ag+i[tt] 7 ^ 00 for non-faulty nodes v,u, regardless 
of whether they execute the first instruction of / 3£+2 or not. Trivially, dq+i[v] = 1 at each 
non-faulty node v due to the second instruction of D 

Next, we argue that once agreement is established, it persists—it does not matter any more 
which instruction sets are executed. 

Lemma 5. Assume that aq[v\ = x G [C] and dg[v] = 1 for all non-faulty nodes v in some 
round q. Then aq+i[u] = x -|- 1 mod c and dgj^i[v\ = 1 for all non-faulty nodes v. 

Proof. Each node will observe at least N — F nodes with counter value x, and hence at most F 
nodes with some value y ^ x. For non-faulty node v, consider all possible instruction sets it 
may execute. 

First, consider the case where instruction set is executed. In this case, v increments x, 
resulting in ag+i[v\ = x -|- 1 mod C and dq+i[v] = 1. Second, executing Ise-ei^ node v evaluates 
Zx > N — F and Zy < F for all y x. Hence it sets dg+i [u] = 1 and Og+i [u] = x -|- 1 mod C. 
Finally, when executing / 3 £+ 2 , node v skips the first instruction and sets (iq+i[u] = 1 and 
Uq+i [u] = X -|- 1 mod C. □ 

3.5 Proof of Theorem 1 

We can now prove the main result. As shown in Lemma 3 we have constructed a r-counter that 
will remain consistent at least r rounds. This is a snfhciently long time for the nodes to execute 
the phase king protocol in synchrony. This protocol will stabilise the C-counter for the network 
of N nodes. More precisely, each node {i,j) runs the following algorithm; 

1. Update the state of algorithm A*. 

2 . Compute the counter value R. 

3. Update state according to instruction set Ir of the phase king protocol. 

By Lemma 3, there is a round t < T(A) -\- — t so that the variables Rq[i,j] meet the 

requirements of r-counting for rounds t < q < t -\- t. For each round q, all non-faulty nodes 
execute the same set of instructions. In particnlar, as r = 3{F -\- 2), no matter from which value 
the r-counting starts, for at least F -|- 1 values £ G [F + 2] the instruction sets I^i, Isi+i, and 
/ 3 £_i_ 2 , in this order, will be jointly executed by all non-faulty nodes at some point during rounds 
t,... -I- r - 1. 

As there are only F faulty nodes, there are at least two non-faulty nodes £ G [F -|- 2]. Thus, 
the prerequisites of Lemma 4 are satisfied in some round q < t -£ t < F(A) -|- c^. By an inductive 
application of Lemma 5, we conclude that the variables ag[v\ are valid outputs for C-counting, 
and therefore, we have indeed constructed an algorithm B G A{N, F, C). 

The bound on q yields that F(B) < r(A) Ck = T{A) -\- 3(F -|- 2){2m)^. Concerning the 
state complexity, observe that each non-faulty node (f, j) needs the memory for execnting ( 1 ) the 
algorithm Aj, which needs S{Ai) = S{A) bits of memory, and (2) the phase king protocol, which 
needs |'log(C' -|- 1)] bits to store ag[v] G [C] U { 00 } and one additional bit to store dq[v\. □ 
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Figure 2: Recursive application of our construction using k = 3 blocks. The filled circles represent 
the nodes. The groups of four nodes run 1-resilient counters. On top of this, the dashed groups 
run 3-resilient counters. At the top-most layer, the nodes run a 7-resilient counter. Faulty blocks 
are drawn with a red border, whereas faulty nodes are marked with a red cross. 

4 The Recursive Construction 

In this section, we show how to use Theorem 1 recursively to construct synchronous c-counters 
with a near-optimal resilience, linear stabilisation time, and a small number of states (see Figure 2 
for an illustration). First, we show how to satisfy the preconditions of Theorem 1 in order to 
start the recursion. Then we demonstrate the principle by choosing a hxed value of k throughout 
the construction; this achieves a resilience of for any constant e > 0. However, as the 

number of nodes in the initial applications of Theorem 1 is small, better results are possible by 
starting out with large values of k and decreasing them later. This yields an algorithm with a 
resilience of and 0 {f) stabilisation time using 0(log^ //loglog / -|- logc) state bits. 

4.1 The Base Case 

To apply Theorem 1, we need counters of resilience / > 0. For example, one can use the 
space-efficient 1-resilient counters from [5] as base of the construction. Alternatively, we can 
use as a starting point trivial counters for n = 1 and / = 0. Then we can apply the same 
construction as in Theorem 1 with the parameters n = 1, / = 0, A: = V, and F < N/3. The 
same proof goes through in this case and yields the following corollary. Note that here the 
resilience is optimal but the algorithm is inefficient with respect to the stabilisation time and 
space complexity. 

Corollary 1. For any c > 1, there exists a synchronous c-counter with optimal resilience 
f < n/3 that stabilises in rounds and uses 0{f log f + logc) bits of state. 

Proof. For any / > 0, we can construct a /-resilient counter for 3f + 1 nodes. We use the trivial 
0-resilient counter for one node as the base case for Theorem 1 and set k = 3f + 1, that is, 
each block consists of a single node. Theorem 1 yields an /-resilient algorithm that stabilises in 
(3/ -I- 2)(2m)^ = rounds, where the 0{f) term is (3 -|- o(l))/. □ 

4.2 Using a Fixed Number of Blocks 

For the sake of simplicity, we will first discuss the recursive construction for a fixed value of k 
here. Improved resilience can be achieved by varying k depending on the level of recursion which 
we show afterwards. 

Theorem 2. Let 1 > e > 0 and 2 < c G N. There exists a synchronous c-counting algorithm with 
a resilience of f = that stabilises in 0(2^^'"^/) rounds and uses log /-|-log^ /-|-logc) 

bits of state per node. 

Proof. Fix 0 < e < 1 and let A: = 2h > 4 be minimal such that e > 1/logh. Assume f = 2^ 
for some j > h and let L = log//log/i; w.l.o.g., assume that L G N. We will analyse how 
many nodes are required to get to a resilience of r2(n^“^) by applying Theorem 1 for L < slogf 
iterations. 
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For all i > 0, let fi = and n* = 4A:*. At iteration i + 1 we use Theorem 1 to construct 
algorithms in A{ni+i, fi+i,c) (for any c) using m and fi as the input parameters. Since 
/i+i = fih < {fi + l)\k/2] and Uj+i = krii, the conditions of Theorem 1 are satisfied. To start 
the recursion, we will use an algorithm with parameters /o = 1 and no = 4. By Corollary 1, such 
an algorithm with a stabilisation time of Tq = 0(1) and a space complexity of Sq = O(logc) 
exists. 

Every iteration increases the resilience by a factor of at least h = k/2. After L iterations, we 
tolerate at least f = fL = failures using n = ni = 4/c^ nodes. This gives 

and it follows that the resilience is n(n^“^). 

It remains to analyse the stabilisation time and space complexity of the resulting algorithm; 
both follow from Theorem 1. The stabilisation time of layer i + 1 is 

Ti+i <Ti + 3(/j+i + 2)h^, 

where h = \k/2']. From the definition of fi, we get the bound J2i=o fi ~ ^(/i) = 0{f). 
Therefore, the overall stabilisation time is 

T = TL<o(^h’^J2j}j =0{h^f). 

From Theorem 1, we get that the space complexity of layer i + 1 is 

Si+i ^ Si + |'log(3(/i+2 + 2)h^ + 1)] + 1 = S'* + 0(A:log h + log/). 

As L < elogf, the total number of memory bits is then bounded by 
L 

0{klogh + log/) + O(logc) < 0{ekloghlog f + log^ / + logc). 

i=0 


Recall that h = k/2 is minimal such that e > 1/log/i. Thus k = 0(2^/^) and log/i = 0{l/e), 
yielding the claimed bounds on time and space complexity. □ 

Choosing a constant £, we arrive at the following corollary. 

Corollary 2. For any constant 1 > e > 0 and any 2 < c G N, there exists a synchronous 
c-counter with resilience f = fl(n^“^) that stabilises in 0{f) rounds and uses 0(log^ n + logc) 
bits of state. 

4.3 Varying the Number of Blocks 

Obviously, the factor 2^^^'" makes the previous construction impractical unless 1/e is small. 
However, it turns out that we can still achieve good resilience without a doubly-exponential 
blow-up in the stabilisation time by carefully varying the number of blocks at each level. 

Theorem 3. For any c> 1, there exist synchronous c-counters with a resilience of f = 
that stabilises in 0{f) rounds and uses 0(log^//loglog/-|-logc) bits of space per node. 

Proving this theorem boils down to choosing k in each iteration as large as possible without 
violating the bound on the stabilization time. We again rely on Theorem 1, but instead of using 
a fixed number of blocks at each iteration, we divide the construction into P phases. During 
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each phase, we use a different number of blocks and iterations of Theorem 1. The goal is to 
have the running time of the last phase dominate the running time of earlier phases. 

We set each phase 1 < p < P to use kp = A ■ 2 ^~p > 3 blocks per layer and then iterate 
Theorem 1 exactly Rp = 2kp times. During phase p, we use iteration 1 < f + 1 < Pp to get an 
algorithm from Theorem 1 that tolerates 

f _ fi,pkp ^ ^ 

h+i,p — 2 ^ -^1 2 

failures, where /o,p = /Hp_i,p-i and /ro,o = /o,o = 1- Thus, for any p and i, the values kp and 
fi^p satisfy the conditions of Theorem 1. Again, to start the recursion we may use any algorithm 
tolerating a single fault among 4 nodes giving /o,o = 1- 

Now, every phase p increases the resilience by a factor of 



As there are total of P phases, this means that the total resilience and number of nodes n are 
given by 

/ =/p = n = n ( ^ ) n = np = 4:Y[kp^. 

p=l P=1 ^ p=l 

In order to get resilience of / > where 8 = o(l), we want to ensure that 

p p-i 

” = 4 2"^ = 4 J] 22-2^ = 22(2^-!) < /^, 

^ p=l j=0 

which is equivalent to 

2(2^ - 1) = 4 • 2^-1 - 2 = ki-2< slogf. 

Hence, it is feasible to choose e = ki/logf. Observe that Ylp=i — 4(2^ — 1) < 2/ci. It follows 
that 

p p 

log / = log'[\{kp/2)^p = 2^kplog{kp/2) = 0(A:i log/ci), 

P=1 p=l 

implying that ki = ©(log//log log/). We conclude that e = 0(1/log log/). 

Let us now analyse the stabilisation time and space complexity of the construction. 

Lemma 6. The algorithm stabilises in 0{f) rounds. 

Proof. To analyse the stabilisation time, we hrst bound the stabilisation time of each phase p 
separately. By Theorem 1, iteration i + 1 of phase p has stabilisation time 

Ti+i^p = Ti^p + O^fipi^p ■ kp^ 

Analogously to the proof of Theorem 2, we again get a geometric series and the total stabilisation 
time of phase p is bounded by 

Rp 

J2T^+l,p = 0iTn^,p) = 0[fp■k^p^). 
i=l 

Therefore, the total stabilisation time is 

p / p 

P=1 \p=l 
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For 1 < p < -P — 4, using the shorthand k = /cp+i, we can bound 

/p+l - (JlY > 9 

fp ' kp^~ “ 22« ■ “ V16/ - 

Thus, we get a geometric series 

P-3 

Y,Tp<2Tp_2 = 0{f), 

p=i 

and since also Tp _2 + Tp_i + Tp = 0{f), the stabilisation time of all phases is bounded by 

0{f). □ 

Lemma 7. Every node uses at most 0(log^//log log/ +log c) bits of memory. 

Proof. By Theorem 1, the number of state bits increases each iteration by ©(logC**) bits, where 

k 

Ci is the counter size needed for iteration i. During phase p, the counter size is 0{f ■ kp^) in 
each iteration. There are exactly Rp = 2kp iterations phase p, and thus the number of bits we 
need is bounded by 

Sp = 0{kp (log f + kp log kp)) = 0(kp log f + k^ log kp) . 

From earlier computations, we know that Y ~ 0{ki) and ki = 0(elog/). Thus, we use 


S = ^Sp = oi^ikplogf + kllogkp 

p=l Yp=1 

( p p 

logf'^kp + Y^ kp log kp 

p=i p=i 

= 0{ki logf + kj logki) 

= 0{e log^ / + log^ /(log e + log log /)) 

= o(2slP) 

V log log// 


bits in total, as e = 0(1/log log/). Storing the output of the resulting c-counter introduces 
additional O(logc) bits. □ 


5 Saving on Communication Using Randomization 

So far we have considered the model where each node broadcasts its entire state every round. In 
the case of the algorithm given in Theorem 3, every node will send S = 0(log^ // log log / + log c) 
bits in each round. As there are 0(n^) communication links, the total number of communicated 
bits in each round is Q{n^S). In this section, we consider a randomised variant of the algorithm 
that achieves better message and bit complexities in a slightly different communication model. 

5.1 The Pulling Model 

Throughout this section we consider the following model, where in every synchronous round: 

1. each processor contacts a subset of other nodes by pulling their state, 

2. each contacted node responds by sending their state to the pulling nodes, 

3. all processors update their local state according to the received messages. 
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As before, faulty nodes may respond with arbitrary states that can be different for different 
pulling nodes. We define the (per-node) message and bit complexities of the algorithm as the 
maximum number of messages and bits, respectively, pulled by a non-faulty node in any round. 

The motivation for this model is that it permits to attribute the energy cost for a message 
to the pulling node. In a circuit, this means that the pulling node provides the energy for the 
signal transitions of the communication link: logically, the link is part of the pulling node’s 
circuitry, whereas the “sender” merely spends the energy for writing its state into the register 
from which all its outgoing links read. 

Our goal will be to keep the number of pulls by non-faulty nodes small at all times. This way 
a small energy budget per round per node suffices in correct operation. By limiting the energy 
supply of each node, we can also effectively limit the energy consumption of the Byzantine nodes. 

5.2 The High-Level Idea of the Probabilistic Construction 

To keep the number of pulls, and thus number of messages sent, small, we modify the construction 
of Theorem 1 to use random sampling where useful. Essentially, the idea is to show that with 
high probability a small set of sampled messages accurately represents the current state of the 
system and the randomised algorithm will behave as the deterministic one. There are two steps 
where the nodes rely on information broadcast by the all the nodes: the majority voting scheme 
over the blocks and our variant of the phase king algorithm. Both can be shown to work with 
high probability by using concentration bound arguments. 

More specihcally, for any constant k > 1 we can bound the probability of failure by ri~^ by 
sampling M = Q{logr]) messages; here g denotes the total number of nodes in the system. The 
idea is to use a union bound over all levels of recursion, nodes, and considered rounds, to show 
that the sampling succeeds with high probability in all cases. For the randomised variant of 
Theorem 1, we will require the following additional constraint: when constructing a counter 
on N = kn nodes, the total number of failures is bounded by E < 3 ^, where 7 > 0 is some 
constant. Since the resilience of the recursive construction is suboptimal anyway, this constraint 
is always going to be satished. This allows us to construct probabilistic synchronous c-counters 
in the sense that the counter stabilises in time T if for all rounds t > T all non-faulty nodes 
count correctly with probability 1 — 

5.3 Sampling Communication Channels 

There are two steps in the construction of Theorem 1 where we rely on deterministic broadcasting: 
the majority sampling for electing a leader block and the execution of the phase king protocol. 
We start with the latter. 

Randomised Phase King. Instead of checking whether at least N — F oi all messages have 
the same value, we check whether at least a fraction of 2/3 of the sampled messages have the 
same value. Similarly, when checking for at least E -|- 1 values, we check whether a fraction 1/3 
of the sampled messages have this value. 

Lemma 8. Let x G [C] U { 00 } and suppose a node samples M values from the other nodes. Then 
there exists Mo{ri,K,'y) = 0(logry) so that M > Mq implies the following with high probability. 

(a) If all non-faulty nodes agree on value x, then x is seen at least 2/3 • M times. 

(b) If the majority of non-faulty nodes have value x, then more than Ij?, ■ M sampled values 
will be X. 

(c) If at least 2/3 • M sampled values have value x, then x is a majority value. 
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Proof. Define 5 = 1 — | and let the random variable X denote the number of x values 
sampled from non-faulty nodes. 

(a) If all non-faulty nodes agree on value x, then 


E[X] = 

^ ^ V Nj 3+7 

As 5 satisfies (1 — 6 ) E[X] > 2/3 • M, it follows from Chernoff’s bound that 


Pr 


X < -M 


< Pr[X <{l-5) E[X]] 


< exp 



2 + 7 

2(3 + 7) 



For sufficiently large Mq{N, k,^) = G{logN) this probability is bounded by N~'^. 

(b) If a majority of non-faulty nodes have value x, then E[A] > ^ As above, by 

picking the right constants and using concentration bounds, we get that 


Pr 


X <-M 
- 3 


< Pr[A < (1 -(5)E[A]] 

< exp E[A]^ 


< exp 



2 + 7 

4(3 + 7) 



< N-^. 


(c) Suppose the majority of non-faulty nodes have values different from x. Defining X as 
the random variable counting the number of samples with values different from x and arguing as 
for (b), we see that 


2 


I 

X > -M 

= Pr 

X < -M 

- 3 


3 


< N-^, 


where again we assume that Mq(N, K,'y) = 0(logAI) is sufficiently large. Thus, X > 2/3 • M 
implies with high probability that the majority of non-faulty nodes have value x. □ 


As a corollary, we get that when using the sampling scheme, the execution of the phase king 
essentially behaves as in the deterministic broadcast case. 

Corollary 3. When executing the randomised variant of the phase king protocol from Section 3 
for rounds, the statements of Lemma 4 o,nd Lemma 5 hold with high probability. 

Proof. The algorithm uses two thresholds, N — F and E + 1. As discussed, these are replaced by 
2j3 ■ M and IfS ■ M when taking M samples. Using the statements of Lemma 8 , we can argue 
analogously to the proofs of Lemma 4 and Lemma 5; we apply the union bound over all rounds 
and samples taken by non-faulty nodes {N — F < rj per round), i.e., over events. □ 


Randomised Majority Voting. It remains to handle the case of majority voting in the 
construction of Theorem 1. Consider some level of the recursive construction, in which we want 
to construct a counter oi N = kn nodes out of k n-node counters. If V <C log 7 / log log 7 , we can 
perform the step in the recursive construction using the deterministic algorithm, that is, pulling 
from all kn nodes. Otherwise, similar to the above sampling scheme for randomised phase king, 
each node will from each block uniformly sample M > Mo( 7 , ^, 7 ) = ©(log 7 ) states. Again by 
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applying concentration bounds, we can show that with high probability, the non-faulty nodes 
sample a majority of non-faulty nodes from non-faulty blocks. Thus, we can get a probabilistic 
version of Lemma 3. 

Recall from Section 3 that 


6* = majority{6[i, j] | J G [n]}, 

B = majority{6* | f G [A:]}, 

R = majority{r[R, j] | j G [n]}, 

where the majority function may output an arbitrary value if there is no majority of non-faulty 
nodes supporting the same value. Analogously to Section 3, we define the following local variables 
at node {i,j) in round q: 

bq[i,j] = majority{6g[f',/] | (i',/) sampled by {i,j) in round g}, 

Bq[i,j] = majority{6*'[i, j] | i' G [k]}, 

Rq[i,j] = majority{r[Rg[i, j],/] | sampled by {i,j) in round q}. 

Here we sample with repetition and the above sets are multisets; this means all samples from a 
block are independent and we can readily apply Chernoff’s bound. 

Lemma 9. Suppose x G [k] is a non-faulty block, Mq = ©(logr/) is sufficiently large, and all 
non-faulty blocks count correctly in round q. If for all non-faulty blocks i and non-faulty nodes 
{i,j) it holds that b[i,j] = x, then with high probability 

1. 6* [i,j] = X for all non-faulty blocks i', 

2. Bq[i,j] = X, and 

3. Rq[i,j] = rq[x,j'] for an arbitrary non-faulty node {B,j') in block x. 

Proof. Consider a non-faulty block (recall that a block is non-faulty if it has at most / faulty 
nodes). Let X denote the number of states of non-faulty nodes sampled from this block by {i,j) 
in round q. As / < n/3, we have that E[X] > M{n — f)/n > 2/3 • M. Applying Chernoff’s 
bound for (5 = 1/4 and choosing sufficiently large M^irf, n) = 0(logr/), we obtain that 

Pi[X < M/2] < Pr[X < (1 - -5)E[X]] < exp^-y E[X]^ < 

Applying the union bound to all nodes and all blocks, it follows that, with high probability, 
non-faulty nodes always sample a majority of non-faulty nodes from non-faulty blocks. The first 
statement follows, immediately yielding the second as a majority of the blocks is non-faulty. The 
third statement now holds because we assume that non-faulty blocks count correctly and x is 
non-faulty. □ 

5.4 Randomised Resilience Boosting 

Define 'P{n, f ,c,rj, n) as the family of probabilistic synchronous c-counters on n nodes and 
resilience /, where probabilistic means that an algorithm P G 'P(n, /, c, g, k) of stabilisation time 
T(P) merely guarantees that it counts correctly with probability 1 — g~'^ in rounds t > r(p). 
This means that with high probability, eventually all non-faulty nodes agree on a common clock 
for sufficiently many rounds. Together with Corollary 3, we obtain a randomized variant of 
Theorem 1. 
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Theorem 4. Given n, f,r] £ N, pick new parameters N,F,C G N and k > 0, where 

• the number of nodes N = kn < rj for some number of blocks 3 < k G N, 

• the resilience F < (/ + l)m, where we abbreviate m = \k/2\, 

• C > 1 is the new counter size, and 

• K is a constant. 

Choose any c G N that is an integer multiple of 3{F + 2)(2m)^. Then for any A G A{n, f, c), 
there exists P G V{N, F, C, t], n) with the following properties. 

1. r(P) = r(A) + 3(F + 2)(2m)*^, and 

2. S(P) = S{A) + riog(C + 1)1+1. 

3. Each node pulls 0{klogr]) messages in each round. 

Note that we can choose to replace A G A{n,f,c) by Q G V{n, f,c,ri, k) when applying 
this theorem, arguing that with high probability it behaves like a corresponding algorithm 
A G A{n, f, c) for polynomially many rounds. Applying the recursive construction from Section 4 
and the union bound, this yields Corollary 4. By always choosing k = O{log rj), each node pulls 
0(log^ rj) messages from other nodes for each layer. 

Corollary 4. For any c > 1, there exist probabilistic synchronous c-counters with a resilience of 
f = that stabilise in 0{f ) rounds, use 0(log^ //loglog / + logc) bits of space per node, 

and in which each node pulls 0{logri{\og f / loglog f)^) messages per round. 

We note that it is also possibility to boost the probability of success, and thus the period of 
stability, by simply increasing the sample size. For instance, sampling polylog r] messages yields 
an error probability of in each round, whereas in the extreme case, by “sampling” all 

nodes the algorithm reduces to the deterministic case. 

5.5 Oblivious Adversary 

Finally, we remark that under an oblivious adversary, that is, an adversary that picks the 
set of faulty nodes independently of the randomness used by the non-faulty nodes, we get 
pseudo-random synchronous counters satisfying the following: (1) the execution stabilises with 
high probability and (2) if the execution stabilises, then all non-faulty nodes will deterministically 
count correctly. Put otherwise, we can fix the random bits used by the nodes to sample the 
communication links once, and with high probability we sample sufficiently many communication 
links to non-faulty nodes for the algorithm to (deterministically) stabilise. This gives us the 
following result. 

Corollary 5. For any c > 1, there exist pseudo-random synchronous c-counters with a re¬ 
silience of f = against an oblivious fault pattern that stabilise in 0{f) rounds with high 

probability, use 0(log^//loglog/+ logc) bits of space per node, and in which each node pulls 
0 (log? 7 (log//loglog/)^) messages per round. 

6 Conclusions 

In this work, we showed that there exist (1) deterministic algorithms for synchronous counting 
that have (2) linear stabilisation time, (3) use a very small number of state bits while still 
achieving (4) almost-optimal resilience-something no prior algorithms have been able to do. In 
addition, we discussed how to reduce the total number of communicated bits in the network, while 
still achieving (2)-(4) by considering probabilistic and pseudo-random synchronous counters. 
We conclude by highlighting a few open problems; 
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1. Are there randomised or deterministic algorithms with the optimal resilience of / < n/3 
that use polylog / state bits and stabilise in 0{f) rounds? 

2. Are there deterministic algorithms that use substantially fewer than log^ / state bits? 

3. Are there communication-efficient and space-efficient algorithms with high resilience that 
stabilise quickly in the usual synchronous model? 
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